How Not to Be Frustrated with Neural Networks

N eural networks are very powerful as nonlinear signal processors, but obtained results are often far from satisfactory. The purpose of this article is to evaluate the reasons for these frustrations and show how to make these neural networks successful. The following are the main challenges of neural network applications: 1) Which neural network architectures should be used? 2) How large should a neural network be? 3) Which learning algorithms are most suitable? The multilayer perceptron (MLP) architecture (Figure 1) is unfortunately the preferred neural network topology of most researchers [1], [2]. It is the oldest neural network architecture, and it is compatible with all training softwares. However, it will be shown in the latter part of this article that MLP architectures seldom give positive results. The MLP topology is less powerful than other topologies such as bridged multilayer perceptron (BMLP), where connections across layers are allowed (marked as dotted lines in Figure 2). Both MLP and BMLP architectures, as shown in Figures 1 and 2, have four layers, three input nodes, four neurons in the first hidden layer, three neurons in the second hidden layer, and one neuron in the output layer. Shorthand notation for